Not Applicable.
Not Applicable.
The present invention relates to image filtering, and more particularly, to systems for and methods of spatial image filtering, wherein such systems and methods are adaptable so as to optimize the filtering performance.
There are cases in which images should be spatially filtered with a kernel that automatically adapts its filtering characteristic as a functions of certain xe2x80x9cindexes.xe2x80x9d
Examples of such cases are:
1. In noisy image sequences (e.g., fluoro sequences that are characterized by high photon noise), the filter may be part of a xe2x80x9cspatial/temporalxe2x80x9d filter that filters in three dimensions; in two spatial dimensions and in time. Co-pending U.S. patent application Ser. No. 09/651,535, filed Aug. 29, 2000, entitled xe2x80x9cAdaptive Spatial And Temporal Filter For Noise Reduction In Image Sequences,xe2x80x9d and invented by Enrico Dolazza, describes one embodiment of such a spatial/temporal filter. In such a spatial/temporal filter, the spatial filtering component may vary as a function of how much the overall filter should filter in the spatial domain verses the time domain.
2. In compressing image sequences using algorithms that track the motion of the scene, a blurring (i.e., defocusing) of the area of the image where the motion occurs leads to higher compression ratio and lower compression artifacts. In this case the xe2x80x9cindexxe2x80x9d to which the filter adapts is the interframe degree of motion.
3. There are image acquisition devices in which an anisotropic blurring effect takes place as a fixed function of the image location. For example, consider an x-ray detector that has an absorbing layer thickness that is not negligible, and a surface area that is large relative to the distance from the detector surface to the x-ray focal spot. In such a arrangement, the x-rays passing through the detector near the detector perimeter pass through at a significant angle with respect to an axis that is perpendicular to the detector surface. This angular path through the detector produces a blurring effect in portions of the resulting image that correspond to the detector perimeter. In such a case a correction can be performed with a spatial filter whose kernel is a function of the location within the image.
It is an object of the present invention to provide a method of performing such adaptive filtering as required for the preceding examples. The method can be implemented by a software algorithm that is executed on a computer system, or by a circuit which correlates with real time (i.e., 30 or 60 frames/second) image sequences.
The foregoing and other objects are achieved by the invention which in one aspect comprises a system for adaptively filtering an image including a plurality of image pixels, so as to selectively modify the pixels as a function of one or more filtering indexes. The system includes an index source for providing the filtering one or more indexes. In one embodiment, the index source may include an image analyzer that generates the filtering index as a function of some aspect of the image, such as the location of the pixel to be filtered within the image. In another embodiment, the index source may include an external source, such as a device that tracks the motion in the image scene and provides the index as a function of the motion. In another embodiment, the external source may an image sequence analyzer in a spatial/temporal filter, such that the index provides information as to how much the overall filter should filter in the spatial domain verses the time domain. The system further includes a spatial filter for filtering the image, including a time-invariant section and an adaptive section. The spatial filter includes an adaptive kernel that is responsive to the one or more filtering indexes and is adjustable in real time. The adaptive kernel defines the filtering process that is performed upon each of the image pixels.
In another embodiment of the invention, the time-invariant section produces a plurality of distinct filtered outputs, each of which is characterized by a distinct frequency response. The adaptive section scales each of the distinct filtered outputs with a distinct weighting value so as to produce a plurality of scaled filtered outputs, and the adaptive section further combines the plurality of scaled filtered outputs so as to produce a composite filtered output. Preferably the combination includes simple adding, although other embodiments may include other arithmetic operations to combine the scaled filtered outputs. In one preferred embodiment, each of the distinct weighting values is variable as a predetermined function of the one or more filtering indexes. The variability of the weighting values provides the adaptability of the invention.
In another embodiment of the invention, a control section varies the distinct weights according to the one or more filtering indexes related to the image.
In another embodiment of the invention, the time-invariant section further includes a plurality of distinct filtering sections, each of which is characterized by a distinct frequency response.
In another embodiment of the invention, the adaptive section further includes, for each of the distinct filtered outputs, a multiplier for multiplying the associated filtered output by the weighting value so as to produce the scaled filtered output. The adaptive section also includes an adder for summing the plurality of scaled filtered outputs so as to produce the composite filtered output.
In another embodiment of the invention, the image parameters include one or more of absolute pixel values, pixel values relative to one or more surrounding pixels, an image boundary location, and pixel location relative to the image boundary location.
In another embodiment of the invention, the image analyzer determines the image parameters for each image pixel individually.
In another embodiment of the invention, the kernel further includes a constraint of maintaining a substantially constant noise equivalent bandwidth (NEB).
In another embodiment of the invention, the kernel further includes a constraint of maintaining a substantially constant DC value, so as to produce a substantially constant image intensity.
In another embodiment of the invention, the kernel may be selectably configured so as to provide static operation that is not adaptive in real time.
In another embodiment of the invention, the kernel decomposes the image into multiple frequency bands, such that a set of weighting coefficients applied to data in each of the frequency bands is adaptive in real time.
In another embodiment of the invention, the kernel is characterized by a bandwidth, and the bandwidth may be varied by zero padding.
In another embodiment of the invention, the weighting coefficients are separated by intervals of 2N, where N is an integer.
In another embodiment of the invention, the kernel includes a vertical processing branch for processing vertical scans of the image, and a horizontal processing branch for processing horizontal scans of the image, the vertical processing branch and the horizontal processing branch being substantially separate and independent.
In another embodiment of the invention, each vertical and horizontal processing branch includes a weighting function for modifying pixel data within the branch.
In another embodiment of the invention, the weighting function includes one or more of the following: a multiplier, a look-up table, a state machine, a processor, or a digital network.
In another embodiment of the invention, the weighting function in each branch is adjustable, independent of the other branch.
In another embodiment of the invention, the image analyzer further includes a noise analyzer for distinguishing an image change that derives from image noise, from an image change that derives from a change in a scene depicted by the image.
In another embodiment of the invention, the image analyzer quantifies, in addition to determining, the image parameters.
In another embodiment of the invention, the spatial filter decomposes the image into a plurality of image components, each of the image components including information within a predetermined two dimensional frequency band. The spatial filter also applies one of a plurality of weights to each of the image components so as to produce a plurality of weighted image components. The spatial filter further linearly combines the weighted image components, so as to produce a filtered image.
In another embodiment of the invention, the image components are related such that their respective predetermined two dimensional frequency bands vary geometrically.
In another embodiment of the invention, the values of the weights applied to the image components are responsive to the image analyzer.
In another aspect, the invention comprises a method of adaptively filtering an image including a plurality of image pixels, so as to selectively modify the pixels as a function of one or more filtering indexes. The method includes determining one or more filtering indexes, and filtering the image via a spatial filter having an adaptive kernel that is responsive to the one or more filtering indexes and being adjustable in real time. The kernel defines the filtering process performed upon each of the image pixels.
Another embodiment of the invention further includes determining the filtering indexes for each image pixel individually.
Another embodiment of the invention further includes determining filtering indexes as a function of an aspect of the image.
In another embodiment of the invention, the aspect of the image includes the location of the pixel to be filtered within the image.
In another embodiment of the invention, the aspect of the image includes scene changes between different images within a sequence of images.
Another embodiment of the invention further includes quantifying, in addition to determining, the filtering indexes.